FII RISC-V3.01 CPU Processor on FII-PRX100-S (ARTIX-7, XC7A100T) XILINX FPGA Board Evaluation by Coremark and Dhrystone Benchmarks

V1.1

Fraser Innovation inc

FII RISC-V3.01 on FII-PRX100-S (ARTIX-7, XC7A100T) XILINX FPGA Board Evaluation

Version Control

Copyright Notice:

© 2020 Fraser Innovation Inc ALL RIGHTS RESERVED

Without written permission of Fraser Innovation Inc, no unit or individual may extract or modify part of or all the contents of this manual. Offenders will be held liable for their legal responsibility.

Thank you for purchasing the FPGA development board. Please read the manual carefully before using the product and make sure that you know how to use the product correctly. Improper operation may damage the development board. This manual is constantly updated, and it is recommended that you download the latest version when using.

Official Shopping Website:

https://fpgamarketing.com/FII-PRX100-D-ARTIX-100T-XC7A100T-RISC-V-FPGA-Board-PRX100-D-1.htm

FII mainly uses Coremark and Dhrystone Benchmarks to evaluate the RISC-V3.01 CPU (Central Processing Units) performance on FII-PRX100-S (ARTIX-7, XC7A100T) XILINX FPGA Board (https://fpgamarketing.com/FII-PRX100-S-ARTIX-100T-XC7A100T-Xilinx-RISC-V-FPGA-Board-FII-PRX100-S-1.htm).

Coremark has been EEMBC’s CPU evaluation standard since 2009. EEMBC (Embedded Microprocessor Benchmark Consortium) is a non-profit organization with members including Huawei, Intel, ARM and Analog Devices. EEMBC is an important standard for evaluating embedded processors and compilers [1].

Coremark mainly detects ALU (Arithmetic Logic Unit), memory reference, pipeline and branch operations. It is designed to make it impossible for the CPU to run benchmark tests in advance, thus ensuring its fairness. During the specified test time, Coremark does not allow invoking third-party library, and the results are completely based on the optimization of the compiler and the execution processing time of the CPU. Because Coremark mainly provides testing of the CPU architecture, in order to abandon the superiority of the hardware manufacturing process, the final test results of Coremark will be normalized, that is to say, the final test results will be evenly divided into the system clock and the unit is Coremark/MHz. Coremark’s main code is written in C language, including list processing (find and sort), matrix manipulation (common matrix operations), state machine (determine if an input stream contains valid numbers), and CRC (cyclic redundancy check) [2].

The FII RISC-V3.01 system clock is 50MHz, and the Coremark test score shown in Figure 1 is 3.38 (169/50 Coremark/MHz).

Figure 1 FII RISC-V Coremark

Figure 2 is a screenshot of some other CPU Coremarks provided on the EEMBC website.

Figure 2 Part of the CPU Coremark result of EEMBC

Figure 3 CPU Coremark Comparison

FII RISC-V3.01 is a single-core, a mix of 2-stage and 3-stage pipeline CPU. Figure 3 lists some other single-core CPUs’ Coremark being certified by EEMBC. FII RISC-V3.01 has been highlighted using red strokes. It can be seen that FII RISC-V3.01 Coremark is above the average Coremark among the listed 15 CPUs. For the three CPUs which have obviously higher Coremark, they are STMicroelectronics STM32H72x/73x rev Z (highlight as blue), STMicroelectronics STM32H7B3 rev Z (highlight as blue) and Renesas Electronics RX66T (highlight as blue). From the official manual by STMicroelectronics , STM32H72x/73x rev Z and STMicroelectronics STM32H7B3 rev Z both use Cortex-M7, which has a 6-stage super scalar pipeline. Renesas Electronics RX66T uses RXv3 core, which has improved 5-stage pipeline. Since with more stages of pipeline, undoubtedly the better performance of CPU is, to make the comparison of performance more fair, compared with Texas Stellaris Cortex-M3 (highlight as blue), which is also a 3-pipeline processor as FII RISC-V3.01. Nevertheless, FII RISC-V’s Coremark is greatly larger than theirs, even more than two times. Compared with another processor, Microchip ATSAML21J18B (highlight as blue), which is also a three-stage pipeline, the Coremark of FII RISC-V3.01 is still much higher.To conclude, with the same amount core stages of pipeline constrained, FII-RISCV3.01 performs outstandingly and is favourable.

Dhrystone was also a universally recognized benchmark for evaluating CPU performance before Coremark appeared. It was invented by Reinhold Weicker in 1984. It is collectively called the “classic benchmark” with Livermore, Whetstone and Linpack, and was popular at 70-80s in last century. Each of these four benchmarks has a biased focus. While Dhrystone tests the integer operation for UNIX systems, Whetstone tests the floating point operations for minicomputers, Linpack tests the floating point operations for workstation, and Livermore tests the numeric operations for supercomputers [3]. There are two main versions of Dhrystone, Dhrystone 1.1 and 2.1. Dhrystone2.1 has been improved on the basis of Dhrystone1.1, so that part of the code will not be disabled due to optimization, thereby affecting the accuracy of the evaluation. The main code of Dhrystone is written in C language, and its performance is tested on the CPU’s Millions of Instructions Per Second (MIPS). The final result needs to be normalized and divided by 1757. This is because historically the Dhrystone test result of 1977 Digital Vax 11/780 was 1757 Dhrystone/s (it can execute 1757 times Dhrystone benchmark per second), which was considered the world’s first microcomputer with a computing level of one million instructions per second. Therefore, Dhrystone MIPS (DMIPS) tested on other platforms will be normalized. The limitation of Dhrystone is that it invokes some library functions to perform iteration, and the compiler usually optimizes it and converts it into assembly language for execution, so in fact Dhrystone is also testing the degree of optimization of the compiler to the C library functions of a specific CPU . As shown in Figure 4, the test score of FII RISC-V3.01 Dhrystone 2.1 is 98360 Dhrystone/s, 55.98 DMIPS (98360/1757), and 1.12 DMIPS/MHz (55.98/50MHz).

Figure 4 Dhrystone test results

Figure 5 CPU Dhrystone plot

Figure 5 shows some CPU Dhrystone results provided by ARM official website. FII RISC-V3.01 has been red highlighted. It can be seen that the CORTEX-M7 (marked blue) has the highest Dhrystone among the 10 CPUs. From the ARM official website, CORTEX-M7, CORTEX-R4 (marked blue), and CORTEX-A5 (marked blue) all have more than 3-stage pipelines. CORTEX-R4, and CORTEX-A5 have more than one core. Same as before, to make the comparison of performance more fair, FII RISC-V3.01 is compared with CORTEX-M3 (marked blue), which is also a single-core and has a 3-stage pipeline. From Figure 5, Dhrystone of CORTEX-M3 is slightly higher than FII RISC-V3.01’s. It could be true since during Dhrystone testing, a universal compiler is used to test FII RISC-V3.01. While Dhrystone relies heavily on standard library functions, and compiler vendors usually optimizes these functions in assembly language. In other saying, Dhrystone test largely depends on the level of the optimization of the compiler instead of the actual hardware operation ability [4]. A specialized compiler could significantly improve the Dhrystone result. It can be proved that by experiments that with specially designed compiler, Dhrystone marks could be even 4 times higher [5]. To conclude, FII RISC-V behaves middling when comparing with the CPU that has the same amount of core and stages of pipeline. Besides, Dhrystone results are frequently quoted and used by many people in the industry, but they do not withstand scrutiny. Without authoritative organizations to certify the test result, Dhrystone is sometimes misused. Although, Dhrystone benchmark has its limitation, it is still a good self-estimation tool as a reference.

In addition to Coremark and Dhrystone, there are many other open source benchmarks, such as AIM Multiuser Benchmark, Embench, HINT, etc., and non-free industry standard, such as SPEC (Standard Performance Evaluation Corporation), BAPCo (Business Application Performance Corporation) that provide benchmarks that can be audited and verified [6].

References

1. “EEMBC | Wikiwand”, Wikiwand, 2020. [Online]. Available: https://www.wikiwand.com/en/EEMBC. [Accessed: 29- Sep- 2020].
2. “EEMBC”, Eembc.org, 2020. [Online]. Available: https://www.eembc.org/coremark/index.php. [Accessed: 29- Sep- 2020].
3. “Roy Longbottom’s PC benchmark Collection – Classic Benchmarks”, Roylongbottom.org.uk, 2020. [Online]. Available: http://www.roylongbottom.org.uk/classic.htm. [Accessed: 28- Sep- 2020].
4. “Dhrystone howto – CDOT Wiki”, Wiki.cdot.senecacollege.ca, 2020. [Online]. Available: https://wiki.cdot.senecacollege.ca/wiki/Dhrystone_howto#What_Dhrystone_really_does. [Accessed: 28- Sep- 2020].
5. T. Riemersma, “The Dhrystone benchmark, the LPC2106 and GNU GCC”, Compuphase.com, 2020. [Online]. Available: https://www.compuphase.com/dhrystone.htm. [Accessed: 28- Sep- 2020].
6. “Benchmark (computing) | Wikiwand”, Wikiwand, 2020. [Online]. Available: https://www.wikiwand.com/en/Benchmark_(computing). [Accessed: 29- Sep- 2020].